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Abstract 

If the n x p matrix X is a stationary point of the MDS loss function, then it is also 
the global minimum over the subspace of all n x p matrices with the same column 
space as X. 


Contents 

1 Introduction 1 

2 Main Result 2 


References 3 
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pdf version, the bib hies, and the complete Rmd hie. 


1 Introduction 

In (Euclidean, least squares, metric) multidimensional scaling (MDS) we minimize the stress 
loss function er(«), defined as 

o(X)=\ X <"«(*«- d„(X)) 2 (1) 

over all configurations X e M nxp , the linear space of n x p matrices. 

Here D(X) = {dij(X)} is a matrix of Euclidean distances between the rows of A", i.e. 

<Uj(x) = 

We now introduce some standard MDS notation, following De Leeuw (1977). Define the unit 
vectors e*, which have element i equal to one and all other elements equal to zero. For i < j 
define the matrices 

Aij (e* e_j)(ej efi . 

Note that dij(X) = y4r X'A^X. Next, define the matrix V = {ty,} by 

V=ZE m U A ‘r P) 



l 



( 3 ) 


Also define the matrix valued function B(X) = {bij(X)} by 


B(X)= ££ 

1 <i<j<n 


where 


r«(V) 


dij(X) 

0 


if dij(X) > 0, 
if dij(X) = 0. 


We also assume, without loss of generality, that dissimilarities are normalized as 


\ E E w i$i - L 


Using these definitions and conventions gives 

a(X) = 1 - tr X'B(X)X + ^tr X’VX, 
and if dij(X) > 0 for all i < j 


Va{X) = (V - B{X))X. 

A conhguration is a stationary point if (V — B(X))X = 0. A stationary point is regular if 
dij(X) > 0 for all i < j. De Leeuw (1984) shows that local minima are regular stationary 
points. 


2 Main Result 


Stationary points can be local minimum points or saddle points. The only local maximum 
point of stress is at X = 0 (De Leeuw (1993)). Among the local minimum points there are 
one or more global minimum points. A sufficient condition for a local minimum to be global 
is that at the stationary point we have V — B(X ) > 0, or V + B(X) < / (De Leeuw (2016)). 
This is a very restrictive condition which we generally do not expect to to be true. There is, 
however, a much weaker relation between stationary points and global minima on a subspace. 


Theorem 1: 

then 


[Local-Global] If X e M nxp is a regular stationary point of the MDS problem 


min cr(XT) 

TeRpxp 


a(X). 


Proof: First, observe that 

dij(XT) = y/ir X'AijXTT' 

which is the square root of a non-negative linear function of S' = TT' , and is consequently 
concave in S. It follows that 


a(XT) = 1 - tr X:'B{XT)XS + ^tr X'VXS 


is convex in S. From Rockafellar (1970), theorem 31.4, the minimum over S > 0 is attained 
at a unique point where 
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1. S>0. 

2. X\V - B(XT))X > 0. 

3. tr X'(V - B(XT)X)XS = 0. 

But if X is a stationary point of the MDS problem we have ( V — B(X))X = 0. Thus the 
minimum over S is attained at S — I, and the minimum over T is attained at any rotation 
matrix T with T'T = XT' = I, which is what the theorem says. ■ 

The part of theorem 1 where it is shown that cr(XT) has a unique (and thus global) minimum 
over T for fixed X is mentioned in Borg and Groenen (2005), p 283. I merely added the result 
that the unique minimizer T is necessarily a rotation matrix if X is a stationary point. Note 
that the stationary point X may be a saddle point, it does not have to be a local minimum 
point. 

An important special case of the theorem is full-dimensional scaling (De Leeuw (1993), De 
Leeuw, Groenen, and Mair (2016)), in which p = n. 

Corollary 1: [Full] If X G M nxn is a stationary point of the MDS problem then it is the 
unique global minimum. 

Proof: In this case 

min a (XT) = min cr(Z). 

TeK" Xn zeR nx p 

By theorem 1 consequently at a stationary point X 

a(X) = min a(Z ). 

Z£«.rixp 
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